Predicting Classifcation Decisions with Data Point Based Meta-learning

Meta-learning involves the construction of a classifier that predicts the performance of another classifier. Previously proposed approaches do this by making a single prediction (such as the expected accuracy) for a complete data set. We suggest modifying this framework so that the meta-classifier predicts for each data point in the data set whether a particular base-classifier will classify it correctly or not. While this information can be converted into a standard meta-learning output such as an overall accuracy estimate for the complete data set, the approach has the added advantage of providing more fine-grained information which promises to be useful in Multiple Classifier Selection and Semi-Supervised Learning. This paper describes the new framework and reports the results of an initial evaluation on a medium-sized database of classification data sets.